Touch Screen Testing Platform for Engaging a Dynamically Positioned Target Feature

ABSTRACT

A touch screen testing platform may be used to engage a dynamically positioned target feature being displayed on a touch screen enabled device during a testing protocol. The platform may record imagery displayed by the touch screen device and then analyze the imagery to locate the target feature within a reference coordinate system. The platform may recognize that the target feature is missing from the imagery and respond by causing the touch screen device to scroll through a command menu and/or toggle through virtual screens. Once located, the platform may instruct a robotic device tester to select the target feature by contacting the touch screen at the identified location using a conductive tip designed to simulate a user&#39;s fingertip. Prior to running a test, the camera may be focused to a point that is offset from the display screen of the touch screen device.

CROSS REFERENCE TO RELATED APPLICATIONS

This patent application is a Continuation-in-Part of co-pending U.S.patent application Ser. No. 14/314,339 entitled “Touch Screen TestingPlatform having Components for Providing Conductivity to a Tip,” filedon Jun. 25, 2014, which application is a Continuation-in-Part of U.S.patent application Ser. No. 12/964,427 entitled “Touch Screen TestingPlatform,” filed on Dec. 9, 2010, which are each hereby incorporated byreference in their entirety as if fully set forth herein.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to robotically testing touchscreen devices and, more particularly, it relates to a testing systemfor dynamically recognizing a location of a target feature on a touchscreen and engaging the touch screen at that location to select thetarget feature.

BACKGROUND

The electronics industry is a dynamic industry where new products arecontinually being released and implemented for use by people andbusinesses in the marketplace. Many new products include touch screensthat enable a user to input commands to an electronic device by touchingthe screen of the device rather than relying on traditional inputs suchas buttons and directional control pads.

Before a product (e.g., device, system, software, and/or hardware) isimplemented in the market or made available for consumption, the productoften undergoes testing to ensure that the product is fully functionaland operational upon deployment. The testing may be used to measuredurability, battery performance, application performance, screensensitivity, or other quantifiable aspects of the operation of theelectronic device subjected to the testing.

Traditional testing platforms are configured to test telecommunicationdevices that have traditional inputs, such a buttons, which have a fixedlocation on the device. However, with touch screen enabled devices, anapplication designer may place input controls anywhere within thedisplay screen, which may require user interaction to determine alocation of an input control used to perform a desired action.Furthermore, in some instances the location of the input controls is maychange over time. Accordingly, the positions of some input controlscannot be assumed to be statically located each time they are displayed.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description is set forth with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical items or features.

FIG. 1 is an illustrative environment including a robotic device testerand a controller to identify a location of a target feature displayed ona touch screen device and control movement of the robotic device testerto selectively engage the touch screen at the location of the targetfeature.

FIG. 2 depicts various imagery displayed by a touch screen device overtime in response to various inputs received from the robotic devicetester.

FIG. 3 is a flow diagram of an illustrative process to perform testingon the touch screen device that includes robotically engaging the targetfeature at a location thereof that is identified based on a camera feed.

FIG. 4 is a flow diagram of an illustrative process to perform testingon the touch screen device that includes determining whether the targetfeature has a known/static location or an unknown/dynamic location.

FIGS. 5A-5C depict the platform in various states of a cameracalibration procedure, in accordance with various implementations.

FIG. 6 is a flow diagram of an illustrative process to calibrate acamera to have a focal offset from the touch screen device during orprior to a testing protocol.

DETAILED DESCRIPTION

A touch screen testing platform may be used to perform repeatabletesting of a touch screen device, such as a telecommunications devicethat includes a touch screen display. During a test scenario, a roboticdevice tester may initiate various operations of the touch screen deviceby engaging a touch screen of the touch screen device. The operations inthe test scenario may include, without limitation, initiating voicecalls, transmitting and receiving data (messages, videos, music, etc.),running applications, and performing other operations. By runningscenarios such as the example test scenario described above, touchscreen devices may be tested in a laboratory environment using anautomated process and include relatively quick cycle times, making thetests relatively inexpensive and repeatable. Results of the tests may beanalyzed to determine performance of the touch screen device, which maybe compared to threshold performance metrics or used for other purposes.

Prior to running a test, the platform may be calibrated to determine aplanar surface defined by the touch screen and to establish a coordinatesystem across the planar surface. The controller may then be programmedto interact with the touch screen at known input locations using thecoordinate system. In some instances, however, particular inputlocations may be unknown until actually displayed by the touch screendevice. In such instances, the controller may utilize a camera feed todetermine a presence/absence of and, ultimately, one or more inputlocations, e.g. a location of a target feature corresponding to a devicecommand. For example, the controller may cause the touch screen deviceto display a command and/or navigational menu in which input locationsfor various features may vary and then selectively engage a targetfeature wherever it is displayed. Furthermore, the controller mayrecognize an absence of the target feature and, in response thereto,cause the robot to alter the imagery displayed by the touch screendevice to find the target feature, e.g. by scrolling through thenavigational and/or command menu.

In various instances, the camera may be intentionally focused at a planethat is offset from the planar surface defined by the touch screen. Thecamera may then record imagery displayed by the touch screen, while afocal point of the camera is offset from the touch screen, and providethis imagery to the controller to identify a location of the targetfeature on the touch screen.

It should be understood that although the disclosure describes severalexamples and related embodiments, the disclosure is not intended to beall-inclusive nor exhaustive in its descriptions. As such, it should beappreciated that the subject matter of the disclosure can be reasonablymodified, rearranged, or otherwise altered, to achieve similar results.

FIG. 1 is an illustrative environment 100 including a robotic devicetester (“robot”) 102 and a controller 104 to identify a location of atarget feature 106 displayed on a touch screen enabled electronic device(“touch screen device”) 108 and control movement of the robot 102 toselectively engage the touch screen device 108 at the location of thetarget feature 106. The robot 102 may operate in accordance withinstructions that are received from the controller 104. For example, thecontroller 104 may transmit a command to the robot 102. In response tosuch commands, the robot 102 may then execute a movement to cause a tip138 of a moveable arm to engage a touch screen display of the touchscreen device 108 and thereby initiate an operation to be performed bythe touch screen device 108 (e.g., initiate a telephone call, interactwith an application, etc.).

The illustrative environment also includes a camera 110 to recordimagery displayed by the touch screen device 108 and transmit theimagery (e.g. as a video feed and/or individual still images) to thecontroller 104 to locate the target feature 106 on the touch screendevice 108 and, after the target feature location becomes known, totransmit a command to the robot 102 to select the target feature 106.Accordingly, when a particular target feature is dynamic such that itmay be displayed at/on various locations and/or virtual screens of thetouch screen device 108, the controller 104 may locate such targetfeatures, e.g. by analyzing imagery from the camera 110, as a pre-curserto instructing the robot 102 to engage these target features. When aparticular target feature is displayed at a static location (e.g. afeature that, when displayed, is always displayed at a known location)the controller 104 may instruct the robot 102 to engage the targetfeature at the known location without searching for or otherwiselocating the target feature.

As illustrated, the controller 104 may be equipped with one or moreprocessor(s) 112 and memory 114. The memory 114 may includeapplications, modules, and/or data. In some embodiments, the memory 114may include a platform manager 116 to interact with the robot 102 andthe camera 110. The platform manager 116 may include a calibrationmodule 118, a test protocol module 120, a feature definition module 122,a feature locator module 124, and a tip actuation module 126, amongother possible modules that enable the controller 102 to interact withthe robotic device 104, and thereby perform test scenarios on the touchscreen device 108. Each module is discussed in turn.

The calibration module 118 may be used to calibrate operation of therobotic device 104. In some embodiments, after the touch screen device108 is securely mounted to a testing fixture, the controller 104 mayidentify and store various locations of the touch screen display as partof the calibration operation. For example, the controller 104 mayidentify and store information defining the perimeter of the touchscreen display portion of the touch screen device 108. The calibrationmodule 118 may identify a planar surface of the touch screen display andmay create a reference coordinate system within the planar surface toenable a user (e.g., engineer, researcher, etc.) to designate locationsof various touch screen inputs. For example, the user may designatelocations of virtual buttons representative of a QWERTY keyboard thatare displayed by the touch screen device 108.

The test protocol module 120 may generate and transmit instructions thatcontrol movement of the robot 102, which performs one or more tests byinteracting with the touch screen device 108. The test protocol module120 may provide instructions to perform stress testing, repetitivetesting, performance testing (e.g., speed, battery life, etc.), screensensitivity testing, or other types of testing. The test protocol module120 may store high-level instructions which lack at least someinformation that is used during execution by the robot 102. A high-levelinstruction may be stored which causes the robot 102 to select aparticular target feature, e.g. the “Pause Logging” target feature 106.During execution, such a high-level instruction may execute one or moresub-instructions to obtain any information to be used during executionof the high-level instruction. For example, a high-level instructiondesigned to engage the “Pause Logging” target feature 106 mayinclude/call upon sub-instructions to determine whether the targetfeature 106 is currently displayed on the touch screen device 108 and,if it is, to determine its current location of display. If it isdetermined that the target feature 106 is not currently displayed, thecontroller 104 may cause the robot 102 to provide an input to the touchscreen device 106 which causes the target feature 106 to becomedisplayed, e.g. the robot 102 may provide a “swipe” input across thetouch screen to toggle between virtual screens and/or open a commandmenu.

The feature definition module 122 may store information corresponding tovisual characteristics of various features to be designated as targetfeatures in various instructions implemented or generated by the testprotocol module 120. For example, information corresponding to visualcharacteristics of the “Pause Logging” target feature 106 may describe apattern having two upright bars superimposed over a contrasting circularbackground, e.g. item 136 in FIG. 1. Such visual characteristicinformation may also include definitions of: colors and/or ranges ofcolors for the components of the target features; aspect ratios ofportions of the target feature to other portions, e.g. the size of theupright bars with respect to the circle; an orientation of the targetfeature with respect to the touch screen; and/or any other visualcharacteristics which may be utilized to describe a target feature. Insome embodiments, the feature definition module 122 may be accessiblevia a user interface (“UI”) 128 that is displayed on a monitor 130. TheUI 128 may enable a user to enter defining characteristics for targetfeatures. For example, a user may cause the touch screen device 108 todisplay the target feature 106 while within a field-of-view 132 of thecamera 110 which may transmit a recording of the imagery displayed onthe touch screen device 108 to the feature definition module 122 fordisplay on the UI 128. The UI 128 may enable a user to select a featureof interest for potential designation as a target feature during atesting protocol. For example, a user may manipulate a box 134 to definea perimeter of the feature of interest. Contents of the box 134 assuperimposed over the image of the touch screen may be enlarged in anextracted image box 136 in which a user may make further manipulationsof the image prior to saving it as a target feature for storage in thefeature definition module 122. Exemplary manipulations may include, butare not limited to, removing portions of the image which are unrelatedto the feature of interest, e.g. a background and/or theme color of thetouch screen device 106, or defining characteristics of the targetfeature, e.g. if a stop symbol is being designated as a target featurethen the user may indicate that the feature will always appear as someshade of red. In accordance with various embodiments, the UI 128 mayalso enable the user to interact with the various other modules of theplatform manager 116 including, but not limited to, any of thosespecific modules described herein.

During the running of a testing protocol, the feature locator module 124may scan imagery received from the camera 110 to locate the targetfeature 106 and to provide the determined location data, e.g.coordinates of the target feature in a reference coordinate system, tothe test protocol module 120 to supplement the high-level instructions.For example, the testing protocol module 120 may initiate a high-levelinstruction to pause a logging task being performed by the touch screendevice 108 and may call upon the feature locator module 124 to locatethe target feature 106 so that it can be engaged. The feature locatormodule 124 may then access data stored and/or generated by the featuredefinition module 122 to scan the imagery for the defining visualcharacteristics of the target feature 106. If the visual characteristicsare identified, the feature locator module 124 may generate the locationdata. The feature locator module 124 may also review contextualinformation within the imagery to confirm that the target feature 106has not been misidentified. For example, upon locating a pause symbolsubstantially matching the visual characteristics of target feature 106,the feature locator module 124 may scan the contextual surroundings ofthe identified pause symbol to ensure that it is located next to thephrase “Pause Logging” and that it is not next to the phrase “PauseMusic” or simply superimposed over a streaming media display indicatingthat selection thereof will pause a video rather than the logging task.The feature locator module 124 may also convert imagery received fromthe camera 110 into text using optical character recognition (OCR). Insome embodiments, the feature locator module 124 may also identifyvarious objects, such as virtual buttons, links, or commands that aredisplayed by the touch screen device and may be interacted with usingthe robotic device 102.

The tip actuation module 126 may select and move a tip 138 to engage thetouch screen display of the touch screen device 108. The tips may besynthetic pads (e.g., rubberized, plastic, etc.), that are moveablycontrolled by the robot 102 to engage the touch screen display inaccordance with instructions from the test protocol module 120. In someembodiments, the tip actuation module 126 may select a tip from multipleavailable tips. In various embodiments, the tip actuation module 126 maybe used to controllably perform multi-touch operations on the touchscreen device 108 by moving two or more tips that simultaneously engagethe touch screen display.

Although FIG. 1 only shows one controller, the components/modules of thecontroller 104, or portions of the components/modules may be implementedon separate computing devices, such as a separate computing devicededicated to the robot 102 and a separate computing device dedicated torunning testing protocols. Thus, the various components described inFIG. 1 may be implemented, in whole or in part, across any combinationof different computing devices in accordance with this disclosure.

FIG. 2 depicts an exemplary implementation 200 in which various imageryis displayed by a touch screen device 108 in response to various inputsreceived from the robot 102 over time. In particular, FIG. 2 depicts thetouch screen device 108 changing displayed imagery responsive to variousinputs provided to the touch screen via robotic movement of the tip 138with reference to a timeline 202. The timeline 202 starts at the leftsuch that time lapses from left to right.

In the exemplary implementation, the touch screen device 108 is depictedas displaying a “Begin Logging” target feature 106(1) which may beselected to cause a logging application to generate a log of datacorresponding to the inputs received or actions performed by the touchscreen device 108 during the testing protocol. At T₀ the robot 102 maymove the tip 138 into contact with the touch screen to initiate thegeneration of the data log.

During the time interval spanning from T₁ to T₂, the platform mayperform a portion of a testing protocol while the touch screen device108 generates a log thereof. For example, the testing protocol mayinclude confirming that the touch screen device 108 is capable ofreceiving messages and/or phone calls. At some point during the testingprotocol, it may be desirable to engage another target feature, e.g.“Pause Logging” target feature 106(2) but the location of this targetfeature may not be ascertainable until the target feature 106(2) isactually visible on the touch screen. Furthermore, in someimplementations one or more inputs may be required in order to cause thetarget feature 106(2) to be displayed. Accordingly, at T₂ the robot 102may move the tip 138 into contact with the touch screen to cause thecause the target feature 106(2) to be displayed, e.g. the tip 138 may beswiped across the touch screen to cause a command menu to be displayed,e.g. the command menu as displayed on UI 128 of FIG. 1 and in variousdisplays portions of FIG. 2.

During the time interval spanning from T₃ to T₄ , the platform mayidentify the location of the target feature 106(2) and use the locationto supplement a high-level instruction, e.g. an instruction to pause thelogging task. For example, the platform manager 116 may access visualcharacteristic data generated using the feature definition module 122and provide this data to the feature locator module 124 for use inscanning imagery received from the camera 110. Upon determining alocation of and/or coordinates for the target feature 106(2), thefeature locator module may provide this information to the test protocolmodule 120 which then may convert the high-level instruction of pausingthe logging task into detailed instructions which are understandable bythe robot 102, e.g. G-code (or other numerical control programminglanguage) instructing the robot 102 to move the tip 138 into contactwith the touch screen at the current location of the target feature106(2). Then, at T₄ the specific instruction robotic instructions may betransmitted by the controller 104 to the robot 102 thereby causing therobot 102 to selectively engage the target feature 106(2).

During the time interval spanning from T₄ to T₅, the platform mayperform additional portions of the testing protocol until a time when itmay be desirable to engage another target feature 106(3), e.g. a “ResumeLogging” target feature. In some implementations, the platform mayperform a similar action to that which was performed at T₂ to cause thecause the target feature 106(3) to be displayed. However, it may then bedetermined that although the command menu is being displayed the “ResumeLogging” target feature 106(3) is not currently being displayed.Accordingly, at T₅ the platform may respond to this determination bycausing the touch screen device 108 to change the imagery until thetarget feature is displayed, e.g. by swiping the tip 138 up from thebottom of the screen to scroll through the entire command menu until the“Resume Logging” target feature 106(3) is identified by the featurelocator module 124. For example, during the time interval spanning fromT₅ to T₆, the platform may locate and select the target feature 106(3).

FIG. 3 is a flow diagram 300 of an illustrative process to performtesting on the touch screen device that includes robotically engagingthe target feature at a location thereof that is identified based on acamera feed. The process 300 is illustrated as a collection of blocks ina logical flow graph, which represent a sequence of operations that canbe implemented in hardware, software, or a combination thereof. Thecollection of blocks is organized under respective entities that mayperform the various operations described in the blocks. In the contextof software, the blocks represent computer-executable instructions that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,objects, components, data structures, and the like that performparticular functions or implement particular abstract data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described blocks can becombined in any order and/or in parallel to implement the process. Otherprocesses described throughout this disclosure, in addition to theprocess 300, shall be interpreted accordingly.

At 302, the platform may determine instructions corresponding to atesting protocol to cause the robot 102 to engage the touch screen withthe tip 138. The instructions may be designed to simulate userinteractions with the touch screen device 108, e.g. the tip 138 mayemulate conductive properties of a human fingertip and the instructionsmay cause the robot 102 to move the tip (or tips) as a user would moveher fingers to control the touch screen device 108.

At 304, the platform may command the robot 102 to execute a firstinstruction which includes one or more numerical control (NC) commandsto control movement of the tip 138 by the robot, e.g. G-code commandsdefining robotic actuations to achieve precise movement of the tip 138.In some implementations, the execution of the first instruction causesthe touch screen device 108 to display imagery that includes at least atarget feature 106. For example, if the target feature 106 correspondsto a notification and/or command menu then the first instruction mayprompt the touch screen device 108 to open a menu of notificationsand/or commands. Alternatively, if the target feature corresponds to anapplication icon which may appear on one of numerous virtual screens(e.g. a touch screen device operating system may organize applicationicons on multiple virtual screens which a user may toggle through tolocate an icon of interest) then the first instruction may prompt thetouch screen device 108 to toggle to a virtual screen on which thetarget feature is displayed.

At 306, the platform may receive, from the camera 110, recorded imagerywhich includes content displayed by the touch screen device 108. Forexample, the touch screen device 108 may be situated and/or securedwithin a field-of-view of the camera 110 which may transmit a liveimagery feed (e.g. a video feed) to the platform and/or numerous stillimages. As used herein, the term “recording” (e.g. a recording of theimagery) is not limited to recordings which are stored for anyparticular duration of time on a storage medium but rather includes anytransmission of imagery captured by the camera 110.

At 308, the platform may scan or otherwise analyze the imagery receivedfrom the camera 110 to identify the location of the target feature, e.g.coordinates within a reference system used to control the robot. In someinstances, the location may be represented as estimated center point ofthe target feature, e.g. a centroid of the feature. In some instances,the location may be represented as a geometrical perimeter of the targetfeature, e.g. a square or circle defining boundaries of the targetfeature. Identifying the location of the target feature at 308 may bebased on a pattern recognition algorithm to enable the feature locatormodule 124 to access a data file storing pattern information, e.g.visual characteristic information corresponding to a pattern, and usethis data file to scan the imagery and match the pattern information toone or more portions of the imagery. For example, the feature locatormodule 124 may access a data file corresponding to the “Pause Logging”target feature 106(2) and use this file to identify any portions of theimagery which resemble or otherwise match the visual characteristics ofthe “Pause Logging” symbol as displayed in FIG. 2. In someimplementations, the identifying the location of the target feature at308 may utilize machine learning techniques to match patternscorresponding to the data file with portions of the imagery byrecognition of patterns and regularities there between.

Identifying the location of the target feature at block 308, may includeidentifying one or more pixels of the camera imagery that correspond tothe location of the target image. For example, the feature locatormodule 124 may determine that the centroid of the target feature isdisplayed substantially by a particular pixel at a known location in theimagery. The platform may be calibrated such that the relationshipbetween pixel locations to corresponding reference coordinate locationsis known and, therefore, that reference coordinate system locations,e.g. a coordinate system used to program NC commands, may be determineddirectly from pixel location data.

At 310, the platform may use the identified location to cause the robotto engage the target feature on the touch screen, e.g. move the tip intocontact with the touch screen at the identified location. Specific NCcommands may be generated and transmitted to the robot 102 to definespecific tip actuation movements. For example, the robot 102 may becalibrated to define an x-y coordinate system parallel to the touchscreen with an additional z-coordinate being perpendicular to the touchscreen. Accordingly, based on such a coordinate system NC commands (e.g.G-code commands) may be generated to define specific actuations to beperformed by the robot 102.

FIG. 4 is a flow diagram of an illustrative process to perform testingon the touch screen device 106 that includes determining whether thetarget feature has a known/static location or an unknown/dynamiclocation.

At 402, the platform may prompt the touch screen device to display acommand menu and/or a virtual screen. For example, the platform maycause the robot 102 to swipe up or down from the top of the touch screento open a notification/command menu, e.g. as displayed in FIG. 2.Alternatively, the platform may cause the robot 102 to swipe left orright on the touch screen to scroll through various virtual screenswhich display different groupings of features and/or application icons.

At 404, the platform may determine whether the target feature is staticsuch that it is displayed at a constant location on the touch screendevice. For example, the “WiFi ON-Off” target feature 106(4) as shown inFIG. 2 may be static such that anytime the command menu is beingdisplayed the “WiFi ON-Off”button is always displayed on the samelocation on the touch screen. If the target feature is static (e.g.having an input location that is determinable prior to providing theinstruction to open the command menu) then the process may proceed to406 at which the target feature may be engaged at the predeterminedlocation. For example, in a scenario where the testing protocol includesturning the Wifi functionality of the touch screen device 108 on or off,then the platform may proceed from block 402 to block 406 withoutanalyzing any imagery to determine a location of the WiFi ON-Off” targetfeature 106(4) because the location is known before it is evendisplayed.

However, if the target feature is dynamic (e.g. having an input locationthat is undeterminable until it is actually displayed on the touchscreen) then the process may proceed to block 408 at which it isdetermined whether the target feature is currently displayed. Forexample, the platform may scan received imagery to locate the targetfeature. Upon a determination that the target feature is currentlydisplayed, the process may proceed to block 306 as described above. Upona determination that the target feature is not currently displayed, theprocess may proceed to block 410 at which an instruction is executedwhich causes the touch screen device to change the imagery displayed byscrolling through a command menu and/or toggling through various virtualscreens until the target feature is displayed and locatable. Forexample, while performing a testing protocol the touch screen device 106may be instructed (via a command from the robot 102) to download anapplication. The touch screen device 108 may be configured to displayapplication icons in a grid configured to display up to a predeterminednumber of application icons, e.g. in a 4 by 4 grid configuration aparticular virtual screen may display up to 16 application icons. If thetouch screen device 108 currently has a single virtual screen with 15 orless application icons, an application icon for the newly downloadedapplication may be placed on the existing virtual screen and thecontroller 104 may locate it via imagery received from the camera 110.Alternatively, if the existing virtual screen already includes a fullgrid of 16 application icons, then the application icon for the newlydownloaded application may be placed on a newly created virtual screen.In such a case the controller may initially search for the icon on onevirtual screen and, if it is not located on that screen, the controller104 may cause the robot 102 to scroll to the other new virtual screen.

FIGS. 5A-5C depict the platform in various states of a cameracalibration procedure for configuring the camera 110 with an offsetfocal point or plane with respect to the touch screen. For example, invarious implementations the camera 110 may be intentionally out of focuswith respect to the touch screen device. In particular, the actual focalpoint of the camera 110 may be intentionally set either in front of orbehind the touch screen that is displaying the imagery.

Referring to FIG. 5A, a focal jig 502 which includes a focal target 504may be placed at a testing location 506. In some embodiments, thetesting location 506 may include one or more positioning elements 508such as, for example, tapered pegs as shown in FIG. 5. The positioningelements may be configured to mate one or both of the focal jig 502and/or a device fixture 510 for securing the touch screen device 108 atthe testing location. For example, the focal jig 502 depicted in FIG. 5Ais shown to interlock/mate with the same positioning elements 508 withwhich the device fixture 510 interlocks with in FIG. 5C. As illustrated,the distance of the focal target 504 from the camera 110 (see FIG. 5A)is less than the distance from the touch screen of the touch screendevice 108 from the camera (see FIG. 5C). Accordingly, upon causing afocal point of the camera 110 to lie on the focal target 504, e.g.making the image rendered by the camera 110 of the focal target 504 ascrisp as possible, and then switching out the focal jig 502 with thedevice fixture 510, the camera 110 will be intentionally out of focuswith the touch screen device 108 by a focal offset distance 512.

Referring to FIG. 5B, in some implementations, the focal offset distance512 may be achieved without the use of the focal jig 502 but rather byplacing a focal target 514 above the touch screen device 108, e.g.placing a block with a focal target affixed thereto directly on top ofthe touch screen device 108. The focal offset distance 512 achieved bythe implementation shown in FIG. 5B is demonstrated by the light greylines extending from FIG. 5B with the focal target 514 on the screen toFIG. 5C in which the focal target 514 has been removed.

FIG. 6 is a flow diagram of an illustrative process 600 to calibrate acamera to have a focal offset from the touch screen device during orprior to a testing protocol. Process 600 describes steps to be taken toutilize the structures depicted in FIGS. 5A-5C. Other structures mayalso be utilized to achieve the focal offset distance 512.

At 602, a touch screen device may be placed at a testing location withthe field of view of the camera. For example, as depicted in FIG. 5C,the touch screen device 108 may be secured to the device fixture 510which may be positively located at the testing location via thepositioning elements 508.

At 604, the camera may be focused or calibrated to achieve the focaloffset distance 512 from the touch screen device 108. The focal offsetdistance 512 may reduce and/or eliminate distortions of the imageryrecorded by the camera during the testing protocol and, therefore, mayimprove the ability of the feature locator module 124 to identify andlocate coordinates for target features. For example, distortions in theimagery such as Moire lines may impede pattern recognition. The negativeeffects that imagery distortions such as Moire lines may have on patternrecognition techniques may outweigh the negative effects of the camerabeing slightly out of focus, i.e. by the focal offset distance 512. Insome implementations, causing the focal offset distance at 604 takeplace prior to placing the touch screen device at the testing locationat block 602 and may include focusing the camera 110 on the focal target504 of the focal jig 502 at block 606 of process 600. In someimplementations, causing the focal offset distance at 604 take placesubsequent to placing the touch screen device at the testing location atblock 602 and may include placing the focal target 514 on top of thetouch screen device 108 and then focusing the camera 110 on the focaltarget 514 at block 608 of process 600.

Once the focal offset distance 512 has been achieved, acomputer-implemented testing protocol may be initiated at block 610. Forexample, an exemplary computer-implemented testing protocol may includethose operations described with respect to FIGS. 3 and 4 of the presentdisclose.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific structural features and/ormethodological acts described. Rather, the specific structural featuresand/or methodological acts are disclosed as exemplary forms ofimplementing the claims. The scope of the present disclosure andappended claims is not limited by these exemplary forms. In particular,numerous variations, whether explicitly provided for by thespecification or implied by the specification, such as variations instructure features and/or methodological acts, whether now known in theart or subsequently developed, may be implemented by one of skill in theart in view of this disclosure.

What is claimed is:
 1. A method comprising: determining a plurality ofinstructions to cause a robot to move a tip to selectively engage atouch screen of a touch screen device to provide the touch screen devicewith inputs corresponding to a testing protocol, wherein the touchscreen is within a field-of-view of a camera, and wherein the inputssimulate user interactions with the touch screen device; executing afirst instruction, of the plurality of instructions, to cause the robotto provide the touch screen device with a first input, the first inputcausing the touch screen device to display imagery that includes atleast a target feature; receiving, from the camera, a recording of theimagery that includes at least the target feature; identifying, based onthe recording of the imagery, a location of the target feature withinthe field-of-view of the camera; and based on the identifying thelocation of the target feature, executing a second instruction to causethe robot to move the tip to selectively engage the touch screen at thelocation of the target feature thereby providing the touch screen devicewith a second input that is associated with the target feature.
 2. Themethod as recited in claim 1, wherein the first input prompts the touchscreen device to display a command menu that includes the targetfeature, and wherein the location of the target feature depends at leastpartially on a state of the command menu.
 3. The method as recited inclaim 2, wherein the touch screen device is configured to perform alogging task to generate a data log during the testing protocol, andwherein the second input causes the touch screen device to initiate,pause, resume, or terminate the logging task.
 4. The method as recitedin claim 2, further comprising: determining, based on the recording ofthe imagery, an absence of the target feature from the field-of-view ofthe camera; and in response to the determining the absence of the targetfeature, executing a third instruction to cause the robot to provide thetouch screen device with a third input, the third input causing thetouch screen device to scroll through the command menu thereby causingthe imagery to include the target feature.
 5. The method as recited inclaim 1, further comprising: prior to the executing the firstinstruction, causing a graphical user interface of a user terminal todisplay an image of the target feature; and receiving, at the userterminal, a user input that defines one or more boundaries of the targetfeature within the image on the graphical user interface, wherein theidentifying the location of the target feature within the field-of-viewof the camera is based at least on the user input.
 6. The method asrecited in claim 1, further comprising receiving a data filecorresponding to one or more visual characteristics of the targetfeature, wherein the identifying the location of the target featurewithin the field-of-view of the camera is based at least on the datafile.
 7. The method as recited in claim 1, wherein a first set of theplurality of instructions correspond to static input locations that aredeterminable prior to providing the touch screen device with individualinstructions of the first set, and wherein a second set of the pluralityof instructions correspond to dynamic input locations that areundeterminable prior to providing the touch screen device withindividual instructions of the second set.
 8. A system for engaging atarget feature that is dynamically positioned during a testing protocol,the system comprising: a robot to provide inputs to a touch screen of atouch screen device by moving a tip into contact with the touch screen,wherein the touch screen is within a field-of-view of a cameraconfigured to record imagery rendered by the touch screen device; and acontroller to control movement of the tip by the robot by performingoperations of: providing the robot with a first instruction to cause therobot to provide the touch screen device with a first input; receiving,from the camera, a recording of the imagery rendered by the touch screendevice; scanning the recording of the imagery to determine a referencecoordinate location of the target feature within a reference coordinatesystem associated with the robot; and based on the reference coordinatelocation, providing the robot with a second instruction to cause therobot to provide the touch screen device with a second input, the secondinput to simulate a user selection of the target feature.
 9. The systemof claim 8, the operations further comprising: receiving, via a userterminal, user input associated with boundaries corresponding to aplurality of target features within one or more images on a graphicaluser interface; generating, based on the user input associated with theboundaries, target feature data corresponding to visual characteristicsof individual target features of the plurality of target features,wherein the scanning includes accessing a portion of the target featuredata associated with the target feature.
 10. The system of claim 8, theoperations further comprising: determining, based on the scanning, anabsence of the target feature from the field-of-view of the camera; andin response to the determining the absence of the target feature,executing a third instruction to cause the robot to provide the touchscreen device with a third input thereby causing the imagery to includethe target feature.
 11. The system of claim 10, wherein the third inputcauses the touch screen device to scroll from a first set of applicationicons to a second set of application icons, the second set ofapplication icons including the target feature.
 12. The system of claim8, wherein the first input includes at least one of a tap movement toengage the tip to a single location on the touch screen or a slidemovement to drag the tip across a portion of the touch screen.
 13. Thesystem of claim 12, wherein at least one of the tap movement or theslide movement causes the touch screen device to display the targetfeature.
 14. The system of claim 8, the operations further comprising:determining, based on the scanning, pixel location data corresponding toat least one pixel of the recording at which the target image islocated; and determining the reference coordinate location based on thepixel location data.
 15. The system of claim 8, wherein the scanning therecording of the imagery to determine the reference coordinate locationincludes matching visual characteristics of a predetermined pattern to aportion of the recording of the imagery.
 16. A method comprising:placing a touch screen device at a testing location that is within afield-of-view of a camera and adjacent to a robot configured to move atip into contact with a touch screen of the touch screen device; causinga focal point of the camera to be in front of or behind the touchscreen; and initiating a computer-implemented testing protocol to testone or more functionalities of the touch screen device, thecomputer-implemented testing protocol to perform operations including:receiving, from the camera while the focal point is in front of orbehind the touch screen, imagery displayed by the touch screen, theimagery including at least a target feature; identifying, based on theimagery, a location of the target feature on the touch screen; andcausing the robot to select the target feature by moving the tip intocontact with the touch screen at the location.
 17. The method of claim16, wherein the causing the focal point of the camera to be in front ofor behind the touch screen includes: placing a focal jig at the testinglocation, the focal jig including at least one focal target; focusingthe camera onto the at least one focal target; and prior to the placingthe touch screen device at the testing location, removing the focal jigfrom the testing location.
 18. The method of claim 17, wherein theplacing a touch screen device at the testing location includes securingthe touch screen device to a device fixture, the touch screen beingaccessible when the touch screen device is in the device fixture. 19.The method of claim 18, wherein the testing location includes one ormore positioning elements configured to commonly mate with each of thefocal jig and the device fixture.
 20. The method of claim 16, whereinthe causing the focal point of the camera to be in front of or behindthe touch screen includes: placing a focal target above the touch screendevice; and focusing the camera onto the focal target.